FDA has some interesting statistics on the state of quality in our manufacturing arena. In over half of the warning situations and 483s delivered, manufacturing related errors are at the root of the problem.

Janet Woodcock recently spoke out on FDA top priorities for 2015, and, not surprisingly, she is setting a clear top prioritization for the newly minted Office of Pharmaceutical Quality. With stories coming in from all over about 483s being issued due to manufacturing operators forging signatures off on batches they did not truly check, improper cleanroom protocol activities, nodding off on the midnight shift, etc., it is quite clear that quality behavior is in the spotlight. The fact is that drug shortages have made pharmaceutical manufacturing front page news. Ms. Woodcock stated that 54% of drug shortage issues can be tied back to manufacturing errors. But whereas quality metrics are often discussed in terms of rework batches and number of pH and temperature tests performed, what about the very basic behavior of manufacturing operators? Can this be measured, and if so, is it possible to include those metrics in negotiations with our suppliers?

Measuring the quality behavior of manufacturing operators may feel like a psychology curriculum, particularly in this biopharmaceutical industry, dominated as we are by left brain-oriented engineers and scientists. The industry has traditionally shied away from anything suggesting that the measurement of “behavioral” traits could be scientific, or measurable. However, their methodology has been used to measure behavioral traits associated with top notch quality behavior in many other life critical industries, including sterile packaging for food, and airline engine and airplane manufacturing. 

There is a reliable method for CMO and owner companies to measure quality behavior metrics and then include those metrics in supplier agreements, ultimately strengthening the quality of our drug supply chain.

How Does It Work?
The ability to measure quality traits begins with simple psychometric testing of a manufacturing team. Most readers would be familiar with the pre-employment tests often used to match good candidates with the values and specific qualities desired by a hiring company. The approach is the same when testing aptitudes for quality behaviors, such as diligent attention to recording test results, or an appreciation for the necessary timespans between tests, for example. Another example would be to test for the likelihood that an individual would perform at a high level and complete all necessary tasks when no one was watching, as often happens on a midnight shift, where there may be less constant supervision.

This type of psychometric testing has been used in other life critical industries, such as airline engine production, where manufacturing technicians are expected to perform with the idea that what goes up must stay up. Though they will not be a passenger on the plane that they approve for take-off, these technicians know that other lives will be in their hands and they act accordingly, and have a natural inclination to behave in a quality conscious manner.

Characteristics such as analytical skills, focus and systematic reasoning have been measured and then compared with these characteristics in the general public, forming a specific profile for excellence in pharmaceutical quality behavior. Then, training can be used to remediate operators and technicians who are lacking, to move them further along the path of quality desired by that manufacturer. 

The Behavioral Positioning Systemtm (BPS), is a partnership between two companies, both with deep roots inside the pharmaceutical industry. GetReSkilled and cut-e draw on decades of data associated with their online video capability building and assessment testing performed on 14 million employees annually.  

A study was conducted in 2014, with 300 pharmaceutical manufacturing operators and technicians who were considered excellent employees in their pharmaceutical jobs. The study was able to discern very clear differences between the behavior of this group, versus the general public. It was found that the high-performing pharma operators and technicians tend to be systematic, analytical, and focused on immediate tasks and results. They tend to come in lower on the scale, for traits such as acting autonomously, setting overly demanding goals around achievement, or having very high levels of social competence. The study confirmed the systematic behavior around quality and analyzing information, and focusing on tasks, clearly confirmed that these qualities are the ones to measure when setting criteria for training and hiring in the pharmaceutical industry.

The spider diagram above shows the results of the two questionnaires—Shapes and Squares—which are typically administered to start the Behavioral Positioning System. You can see interesting differences between the 300 high performing pharmaceutical workers—blue, pilot study—and the general population.

Here is a typical question that is used during the simple online testing:

Supplier Agreements And Quality Behavioral Metrics
Let’s take the scenario of you, as a supply chain professional, inside a large or medium sized pharmaceutical firm. You regularly seek out qualified CMOs as part of your expansion strategy, and long, arduous contract negotiations ensue. But do you have metrics in your agreement for one of the most important factors—quality behavior on the CMO’s manufacturing team? Are you simply going to trust that their employees have your best quality culture in mind?  You can include scientifically based quality metrics that become the baseline for your expectations, and ask that the CMO manufacturing workforce take simple online evaluations, and then use these results as a point of negotiation and discussion. For example, if you find that the quality behavior traits are rating lower in some individuals, you can now ask for an increased level of training to be included. This is part of the Behavioral Positioning System’s approach to close the loop. Perhaps your agreement can request that a certain percent of the manufacturing team undergo BPS testing. Then what? 

We have found that tying the quality actions of our manufacturing operators to an understanding of the underlying science, i.e. showing the data on which our manufacturing processes are based, is extremely beneficial. Training needs go beyond just signing off that SOP and cGMP training has been delivered, and move into real education showing all employees why SOPs are performed, not just how.

Yet currently, this is not a priority in most GMP programs. And thus, manufacturing errors take place on the floors of our manufacturing facilities.

What if CMOs were asked to test and train for quality behavior? What if smart CMOs incorporated such quality testing of their workforce into their commercial outreach, and set themselves apart from the competition, by, in a sense, proving their commitment to their teams’ quality performance?

Testing can show you where manufacturing teams may be lacking, but often these characteristics can show tremendous improvement if teams are trained to understand why they are performing certain tasks. BPS training uses the science behind the tasks, adopting a theory long held in history education: Don’t just ask students to memorize dates! Instead, show students why history matters to them. In the same way, BPS stresses that a science based approach, delivered in short, simple bites, through 2-5 minute videos can literally alter the approach taken by individuals on a team. In addition, BPS ties the learning to the ultimate patient. The video training shows families—fathers, children, grandparents—who will use the therapies and medicines, thus helping breakdown the mechanical nature of the quality tasks and instead, tie the manufacturing jobs to the ultimate patient. BPS online video training is based on their decades-deep pharmaceutical process knowledge which is used in the actual curriculum at the Dublin Institute of Technology for degreed pharmaceutical operator curriculums. The BPS video capability learning can be delivered in digestible snippets, during typical stand-up meetings prior to shifts while gowning up, immediately before they perform their daily tasks. In addition, BPS can be branded and customized to a company’s process.

Another Way to Look at It: Paradigm Paralysis
A paradigm is a set of practices that define a scientific discipline for a time and provide model problems and solutions for a community of practitioners.

The greatest barrier to a paradigm shift is the inability or refusal to see beyond the current models of thinking. This is called Paradigm Paralysis, i.e. the conviction that the current paradigm in reality tends to disqualify evidence that might undermine the paradigm itself; this in turn leads to a build-up of unrecognized anomalies which continue to grow until they can no longer be ignored and eventually replace the incumbent paradigm with a new one.

The Compliance Culture is the current model of thinking for our industry. Compliance culture implies here that the industry is fixated on having enough paper to show that operators were shown the SOP manual, and took X number of hours of training, that there are Y number of signatures every hour to attest to the quality of the batch in the bioreactor, etc. And, compliance culture, as described here, is the barrier to embracing a paradigm shift to a Patient Centric Quality Culture, in line with making safe and effective medicine for patients. This new paradigm needs to become the solution for manufacturing excellence across the complex and fragile supply chains of our globalized manufacturing industry sector.

Having a patient centric quality culture is all about protecting the life of the patient. Blind repetition of SOPs without understanding why they are to be followed will not benefit the patient in the long run, as short-cuts will always be found with familiarity over time. Real learning about quality behavior goes well beyond senior operators demonstrating SOPs to junior operators at a plant level. Handing the manufacturing workforce SOPs and expecting it to provoke a change in behavior just does not work.

This has not been a priority because the benefits, which assuredly take more time initially, were assumed to be difficult or impossible to measure,  and we all know the mantra that “you can’t manage what you can’t measure.”  So, in order to document what we can measure, our industry has evolved into a compliance culture over time in the mistaken belief that we are protecting the life of the patient because it’s too difficult to measure quality culture, especially across our recently globalized manufacturing industry sector. 

But now, with the Behavioral Positioning System, we can measure more than just training hours and sign-off signatures. We can measure critical quality behavioral traits such as conscientiousness, compliance and analytical skills.   

We can learn from other life critical industries that have measured these critical quality behavioral traits across their workforce—industries such as aircraft maintenance, whose technicians have an unwritten mission statement of “what goes up must stay up.”
Pharma does not have a monopoly on life critical industries, and we can learn from those industries who have gone before us in measuring the conscientiousness of their manufacturing workforce and implementing quality metrics that will strengthen their reliability.

The ability to predict and strengthen the critical behavioral traits of our manufacturing and technician teams  is now possible, and is being included in discussion and agreements between big pharma and its contract suppliers. It can be used by CMOs who want to prove the superior quality performance of their team and gain a competitive advantage. In addition, it is expected that FDA will look positively on those who incorporate specific, measurable quality metrics in their training models for operators, before there are problems, and really, as a preventative model for eliminating the potential for quality problems.

The Behavioral Positioning System allows the industry to see beyond the current models of thinking and move past our current status of Paradigm Paralysis into the brave new world of Paradigm Shift, as envisioned and articulated so clearly by Dr. Janet Woodcock in 2013, as we take the first step upon the road to creating a quality culture inside pharmaceutical companies “from the shop floor to the boardroom.” 

---

Gerry Creaner, CEO of GetReSkilled, has over 30-years of experience in the Life Sciences Manufacturing industry, across a range of technical, managerial and business roles. Currently located in Boston, MA, Mr. Creaner’s travels take him regularly to the three other GetReSkilled locations in Singapore, Puerto Rico, and Ireland. A chemical engineer by profession, Mr. Creaner established a successful engineering consultancy prior to founding GetReskilled, an online video education and learning business focussed on the manufacture of safe and effective medicines for the public, with particular emphasis on developing these competencies within the operator and technician groups across the global supply chain.